An Analysis of Sample Attrition in Panel Data: the Michigan Panel Study of Income Dynamics

An Analysis of Sample Attrition in Panel Data: Michigan Panel Study of Income Dynamics By 1989 the Michigan Panel Study on Income Dynamics (PSID) had experienced approximately 50 percent sample loss from cumulative attrition from its initial 1968 membership. We study the effect of this attrition on the unconditional distributions of several socioeconomic variables and on the estimates of several sets of regression coefficients. We provide a statistical framework for conducting tests for attrition bias that draws a sharp distinction between selection on unobservables and on observables and that shows that weighted least squares can generate consistent parameter estimates when selection is based on observables, even when they are endogenous. Our empirical analysis shows that attrition is highly selective and is concentrated among lower socioeconomic status individuals. We also show that attrition is concentrated among those with more unstable earnings, marriage, and migration histories. Nevertheless, we find that these variables explain very little of the attrition in the sample, and that the selection that occurs is moderated by regression-to-the-mean effects from selection on transitory components that fade over time. Consequently, despite the large amount of attrition, we find no strong evidence that attrition has seriously distorted the representativeness of the PSID through 1989, and considerable evidence that its crosssectional representativeness has remained roughly intact. The increased availability of panel data from household surveys has been one of the most important developments in applied social science research in the last thirty years. Panel data have permitted social scientists to examine a wide range of issues that could not be addressed with cross-sectional data or even repeated cross sections. Nevertheless, the most potentially damaging and frequently-mentioned threat to the value of panel data is the presence of biasing attrition--that is, attrition that is selectively related to outcome variables of interest. In this paper we present the results of a study of attrition and its potential bias in one of the most well-known panel data sets, the Michigan Panel Study of Income Dynamics (PSID). The PSID has suffered a large volume of attrition since it began in 1968--almost 50 percent of initial sample members had attrited by 1989. We study the effect of attrition in the PSID on the means and variances of several important socioeconomic variables--such as individual earnings, educational level, marital status, and welfare participation--and on the coefficients of variables in regressions for these variables. We also examine whether the likelihood of attrition is related to past instability of such behaviors--earnings instability, propensities to migrate or to change marital status, and so on. A companion paper studies the effect of attrition on estimates of intergenerational relationships (Fitzgerald et al., 1997b). An understanding of the statistical issues is important to understanding our approach. We provide a statistical framework for the analysis of attrition bias which shows that the common distinction between selection on unobservables and observables is critical to the development of tests for attrition bias and adjustments to eliminate it. However, we show that selection on observables is not the same as exogenous selection, for selection can be based on endogenous observables such as lagged dependent variables which are observed prior to the point of attrition. We note that the attrition bias generated by this type of selection can be eliminated by the use of weighted least squares, using weights obtained from estimated equations for the probability of attrition, and hence without the highly parametric procedures used in much of the literature. Many of our tests for attrition bias are consequently based on whether lagged endogenous variables affect attrition rates. However, we also conduct an implicit test for selection on unobservables by comparing PSID distributions with those from an outside data source, the Current Population Survey (CPS). We find that while the PSID has been highly selective on many important variables of interest, including those ordinarily regarded as outcome variables, attrition bias nevertheless remains quite small in magnitude. The major reasons for this lack of effect are that the magnitudes of the attrition effect, once properly understood, are quite small (most attrition is random); and that much attrition is based on transitory components that fade away from regression-to-the-mean effects both within and across generations. We also find that attrition-adjusted weights play a small role in reducing attrition bias. We conclude therefore that the PSID has stayed roughly representative through